Modern Machine Learning Tools Research Articles

Drone network for early warning of forest fire and dynamic fire quenching plan generation

Wildfires are one of the most frequent natural disasters which significantly harm the environment, society, and the economy. Transfer learning algorithms and modern machine learning tools can help in early forest fire prediction, detection, and dynamic fire quenching. A group of drones that has high-definition image processing and decision-making capabilities are used to detect the forest fires in the very early stage. The proposed system generates a fire quenching plan using particle swarm optimization technique and alerts the fire and rescue department for a quick action, thereby stop the forest fire at an early stage. Also, the drone network plays a major role to track the live status of forest fire and quenches the fire. ResNet, VGGNet, MobileNet, AlexNet, and GoogLeNet are used to detect the forest fire hazards. The experimental results prove that the proposed technique GoogLeNet-TL provides 96% accuracy and 97% F1 score in comparison with the state-of-the-art deep learning models.

DAGGER: Data AuGmentation GEneRative Framework for Time-Series Data in Data-Driven Smart Manufacturing Systems

As industries transition into the Industry 4.0 paradigm, the relevance and interest in concepts like Digital Twin (DT) are at an all-time high. DTs offer direct avenues for industries to make more accurate predictions, rational decisions, and informed plans, ultimately reducing costs, increasing performance and productivity. Adequate operation of DTs in the context of smart manufacturing relies on an evolving data-set relating to the real-life object or process, and a means of dynamically updating the computational model to better conform to the data. This reliance on data is made more explicit when physics-based computational models are not available or difficult to obtain in practice, as it's the case in most modern manufacturing scenarios. For data-based model surrogates to "adequately" represent the underlying physics, the number of training data points must keep pace with the number of degrees of freedom in the model, which can be on the order of thousands. However, in niche industrial scenarios like the one in manufacturing applications, the availability of data is limited (on the order of a few hundred data points, at best), mainly because a manual measuring process typically must take place for a few of the relevant quantities, e.g., level of wear of a tool. In other words, notwithstanding the popular notion of big-data, there is still a stark shortage of ground-truth data when examining, for instance, a complex system's path to failure. In this work we present a framework to alleviate this problem via modern machine learning tools, where we show a robust, efficient and reliable pathway to augment the available data to train the data-based computational models.
 Small sample size data is a key limitation in performance in machine learning, in particular with very high dimensional data. Current efforts for synthetic data generation typically involve either Generative Adversarial Networks (GANs) or Variational AutoEncoders (VAEs). These, however, are are tightly related to image processing and synthesis, and are generally not suited for sensor data generation, which is the type of data that manufacturing applications produce. Additionally, GAN models are susceptible to mode collapse, training instability, and high computational costs when used for high dimensional data creation. Alternatively, the encoding of VAEs greatly reduces dimensional complexity of data and can effectively regularize the latent space, but often produces poor representational synthetic samples. Our proposed method thus incorporates the learned latent space from an AutoEncoder (AE) architecture into the training of the generation network in a GAN. The advantages of such scheme are twofold: \textbf{(\textit{i})} the latent space representation created by the AE reduces the complexity of the distribution the generator must learn, allowing for quicker discriminator convergence, and \textbf{(\textit{ii})} the structure in the sensor data is better captured in the transition from the original space to the latent space. Through time statistics (up to the fifth moment), ARIMA coefficients and Fourier series coefficients, we compare the synthetic data from our proposed AE+GAN model with the original sensor data. We also show that the performance of our proposed method is at least comparable with that of the Riemannian Hamiltonian VAE, which is a recently published data augmentation framework specifically designed to handle very small high dimensional data sets.

Using domain adaptation for classification of healthy and disease conditions from mobile-captured images of standard 12-lead electrocardiograms

12-lead electrocardiogram (ECG) recordings can be collected in any clinic and the interpretation is performed by a clinician. Modern machine learning tools may make them automatable. However, a large fraction of 12-lead ECG data is still available in printed paper or image only and comes in various formats. To digitize the data, smartphone cameras can be used. Nevertheless, this approach may introduce various artifacts and occlusions into the obtained images. Here we overcome the challenges of automating 12-lead ECG analysis using mobile-captured images and a deep neural network that is trained using a domain adversarial approach. The net achieved an average 0.91 receiver operating characteristic curve on tested images captured by a mobile device. Assessment on image from unseen 12-lead ECG formats that the network was not trained on achieved high accuracy. We further show that the network accuracy can be improved by including a small number of unlabeled samples from unknown formats in the training data. Finally, our models also achieve high accuracy using signals as input rather than images. Using a domain adaptation approach, we successfully classified cardiac conditions on images acquired by a mobile device and showed the generalizability of the classification using various unseen image formats.

Demystifying image-based machine learning: a practical guide to automated analysis of field imagery using modern machine learning tools

Image-based machine learning methods are becoming among the most widely-used forms of data analysis across science, technology, engineering, and industry. These methods are powerful because they can rapidly and automatically extract rich contextual and spatial information from images, a process that has historically required a large amount of human labor. A wide range of recent scientific applications have demonstrated the potential of these methods to change how researchers study the ocean. However, despite their promise, machine learning tools are still under-exploited in many domains including species and environmental monitoring, biodiversity surveys, fisheries abundance and size estimation, rare event and species detection, the study of animal behavior, and citizen science. Our objective in this article is to provide an approachable, end-to-end guide to help researchers apply image-based machine learning methods effectively to their own research problems. Using a case study, we describe how to prepare data, train and deploy models, and overcome common issues that can cause models to underperform. Importantly, we discuss how to diagnose problems that can cause poor model performance on new imagery to build robust tools that can vastly accelerate data acquisition in the marine realm. Code to perform analyses is provided at https://github.com/heinsense2/AIO_CaseStudy.

Neural-network decoders for measurement induced phase transitions

Open quantum systems have been shown to host a plethora of exotic dynamical phases. Measurement-induced entanglement phase transitions in monitored quantum systems are a striking example of this phenomena. However, naive realizations of such phase transitions requires an exponential number of repetitions of the experiment which is practically unfeasible on large systems. Recently, it has been proposed that these phase transitions can be probed locally via entangling reference qubits and studying their purification dynamics. In this work, we leverage modern machine learning tools to devise a neural network decoder to determine the state of the reference qubits conditioned on the measurement outcomes. We show that the entanglement phase transition manifests itself as a stark change in the learnability of the decoder function. We study the complexity and scalability of this approach in both Clifford and Haar random circuits and discuss how it can be utilized to detect entanglement phase transitions in generic experiments.

WCN23-0139 Leveraging modern machine learning tools to predict outcomes of in-patient acute kidney injury

WCN23-0139 Leveraging modern machine learning tools to predict outcomes of in-patient acute kidney injury

Classification of Pap-smear cell images using deep convolutional neural network accelerated by hand-crafted features.

The classification of cells extracted from Pap-smears is in most cases done using neural network architectures. Nevertheless, the importance of features extracted with digital image processing is also discussed in many related articles. Decision support systems and automated analysis tools of Pap-smears often use these kinds of manually extracted, global features based on clinical expert opinion. In this paper, a solution is introduced where 29 different contextual features are combined with local features learned by a neural network so that it increases classification performance. The weight distribution between the features is also investigated leading to a conclusion that the numerical features are indeed forming an important part of the learning process. Furthermore, extensive testing of the presented methods is done using a dataset annotated by clinical experts. An increase of 3.2% in F1-Score value can be observed when using the combination of contextual and local features. Clinical Relevance - Analysis of images extracted from digital Pap-test using modern machine learning tools is discussed in many scientific papers. The manual classification of the cells can be time-consuming and expensive which requires a high amount of manual labor. Furthermore the result of the manual classification can also be uncertain due to interobserver variability. Considering these, any result that can lead to a more reliable highly accurate classification method is considered valuable in the field of cervical cancer screening.

Rare-event Simulation for Neural Network and Random Forest Predictors

We study rare-event simulation for a class of problems where the target hitting sets of interest are defined via modern machine learning tools such as neural networks and random forests. This problem is motivated from fast emerging studies on the safety evaluation of intelligent systems, robustness quantification of learning models, and other potential applications to large-scale simulation in which machine learning tools can be used to approximate complex rare-event set boundaries. We investigate an importance sampling scheme that integrates the dominating point machinery in large deviations and sequential mixed integer programming to locate the underlying dominating points. Our approach works for a range of neural network architectures including fully connected layers, rectified linear units, normalization, pooling and convolutional layers, and random forests built from standard decision trees. We provide efficiency guarantees and numerical demonstration of our approach using a classification model in the UCI Machine Learning Repository.

Seeing beyond the Trees: Using Machine Learning to Estimate the Impact of Minimum Wages on Labor Market Outcomes

We assess the effect of the minimum wage on labor market outcomes such as employment, unemployment, and labor force participation for most workers affected by the policy. We apply modern machine learning tools to construct demographically-based treatment groups capturing around 75% of all minimum wage workers—a major improvement over the literature which has focused on fairly narrow subgroups where the policy has a large bite (e.g., teens). By exploiting 172 prominent minimum wages between 1979 and 2019 we find that there is a very clear increase in average wages of workers in these groups following a minimum wage increase, while there is little evidence of employment loss. Furthermore, we find no indication that minimum wage has a negative effect on the unemployment rate, on the labor force participation, or on the labor market transitions. Furthermore, we detect no employment or participation responses even for sub-groups that are likely to have a high extensive margin labor supply elasticity—such as teens, older workers, or single mothers. Overall, these findings provide little evidence for changing search effort in response to a minimum wage increase.

Machine Learning-Based Mortality Prediction of Patients at Risk During Hospital Admission.

The ability to predict in-hospital mortality from data available at hospital admission would identify patients at risk and thereby assist hospital-wide patient safety initiatives. Our aim was to use modern machine learning tools to predict in-hospital mortality from standardized data sets available at hospital admission. This was a retrospective, observational study in 3 adult tertiary care hospitals in Western Australia between January 2008 and June 2017. Primary outcome measures were the area under the curve for the receiver operating characteristics curve, the F1 score, and the average precision of the 4 machine learning algorithms used: logistic regression, neural networks, random forests, and gradient boosting trees. Using our 4 predictive models, in-hospital mortality could be predicted satisfactorily (areas under the curve for neural networks, logistic regression, random forests, and gradient boosting trees: 0.932, 0.936, 0.935, and 0.935, respectively), with moderate F1 scores: 0.378, 0.367, 0.380, and 0.380, respectively. Average precision values were 0.312, 0.321, 0.334, and 0.323, respectively. It remains unknown whether additional features might improve our models; however, this would result in additional efforts for data acquisition in daily clinical practice. This study demonstrates that using only a limited, standardized data set in-hospital mortality can be predicted satisfactorily at the time point of hospital admission. More parameters describing patient's health are likely needed to improve our model.

Does non-linear factorization of financial returns help build better and stabler portfolios?

A portfolio allocation method based on linear and non-linear latent constrained conditional factors is presented. The factor loadings are constrained to always be positive in order to obtain long-only portfolios, which is not guaranteed by classical factor analysis or PCA. In addition, the factors are to be uncorrelated among clusters in order to build long-only portfolios. Our approach is based on modern machine learning tools: convex Non-negative Matrix Factorization (NMF) and autoencoder neural networks, designed in a specific manner to enforce the learning of useful hidden data structure such as correlation between the assets' returns. Our technique finds lowly correlated linear and non-linear conditional latent factors which are used to build outperforming global portfolios consisting of cryptocurrencies and traditional assets, similar to hierarchical clustering method. We study the dynamics of the derived non-linear factors in order to forecast tail losses of the portfolios and thus build more stable ones.

Predicting high-resolution turbulence details in space and time

Predicting the fine and intricate details of a turbulent flow field in both space and time from a coarse input remains a major challenge despite the availability of modern machine learning tools. In this paper, we present a simple and effective dictionary-based approach to spatio-temporal upsampling of fluid simulation. We demonstrate that our neural network approach can reproduce the visual complexity of turbulent flows from spatially and temporally coarse velocity fields even when using a generic training set. Moreover, since our method generates finer spatial and/or temporal details through embarrassingly-parallel upsampling of small local patches, it can efficiently predict high-resolution turbulence details across a variety of grid resolutions. As a consequence, our method offers a whole range of applications varying from fluid flow upsampling to fluid data compression. We demonstrate the efficiency and generalizability of our method for synthesizing turbulent flows on a series of complex examples, highlighting dramatically better results in spatio-temporal upsampling and flow data compression than existing methods as assessed by both qualitative and quantitative comparisons.

Experimental Assessment of UWB and Vision-Based Car Cooperative Positioning System

The availability of global navigation satellite systems (GNSS) on consumer devices has caused a dramatic change in every-day life and human behaviour globally. Although GNSS generally performs well outdoors, unavailability, intentional and unintentional threats, and reliability issues still remain. This has motivated the deployment of other complementary sensors in such a way that enables reliable positioning, even in GNSS-challenged environments. Besides sensor integration on a single platform to remedy the lack of GNSS, data sharing between platforms, such as in collaborative positioning, offers further performance improvements for positioning. An essential element of this approach is the availability of internode measurements, which brings in the strength of a geometric network. There are many sensors that can support ranging between platforms, such as LiDAR, camera, radar, and many RF technologies, including UWB, LoRA, 5G, etc. In this paper, to demonstrate the potential of the collaborative positioning technique, we use ultra-wide band (UWB) transceivers and vision data to compensate for the unavailability of GNSS in a terrestrial vehicle urban scenario. In particular, a cooperative positioning approach exploiting both vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V) UWB measurements have been developed and tested in an experiment involving four cars. The results show that UWB ranging can be effectively used to determine distances between vehicles (at sub-meter level), and their relative positions, especially when vision data or a sufficient number of V2V ranges are available. The presence of NLOS observations is one of the principal factors causing a decrease in the UWB ranging performance, but modern machine learning tools have shown to be effective in partially eliminating NLOS observations. According to the obtained results, UWB V2I can achieve sub-meter level of accuracy in 2D positioning when GNSS is not available. Combining UWB V2I and GNSS as well V2V ranging may lead to similar results in cooperative positioning. Absolute cooperative positioning of a group of vehicles requires stable V2V ranging and that a certain number of vehicles in the group are provided with V2I ranging data. Results show that meter-level accuracy is achieved when at least two vehicles in the network have V2I data or reliable GNSS measurements, and usually when vehicles lack V2I data but receive V2V ranging to 2–3 vehicles. These working conditions typically ensure the robustness of the solution against undefined rotations. The integration of UWB with vision led to relative positioning results at sub-meter level of accuracy, an improvement of the absolute positioning cooperative results, and a reduction in the number of vehicles required to be provided with V2I or GNSS data to one.

Identification of Neural Network Model of Robot to Solve the Optimal Control Problem

To calculate the optimal control, a satisfactory mathematical model of the control object is required. Further, when implementing the calculated controls on a real object, the same model can be used in robot navigation to predict its position and correct sensor data, therefore, it is important that the model adequately reflects the dynamics of the object. Model derivation is often time-consuming and sometimes even impossible using traditional methods. In view of the increasing diversity and extremely complex nature of control objects, including the variety of modern robotic systems, the identification problem is becoming increasingly important, which allows you to build a mathematical model of the control object, having input and output data about the system. The identification of a nonlinear system is of particular interest, since most real systems have nonlinear dynamics. And if earlier the identification of the system model consisted in the selection of the optimal parameters for the selected structure, then the emergence of modern machine learning methods opens up broader prospects and allows you to automate the identification process itself. In this paper, a wheeled robot with a differential drive in the Gazebo simulation environment, which is currently the most popular software package for the development and simulation of robotic systems, is considered as a control object. The mathematical model of the robot is unknown in advance. The main problem is that the existing mathematical models do not correspond to the real dynamics of the robot in the simulator. The paper considers the solution to the problem of identifying a mathematical model of a control object using machine learning technique of the neural networks. A new mixed approach is proposed. It is based on the use of well-known simple models of the object and identification of unaccounted dynamic properties of the object using a neural network based on a training sample. To generate training data, a software package was written that automates the collection process using two ROS nodes. To train the neural network, the PyTorch framework was used and an open source software package was created. Further, the identified object model is used to calculate the optimal control. The results of the computational experiment demonstrate the adequacy and performance of the resulting model. The presented approach based on a combination of a well-known mathematical model and an additional identified neural network model allows using the advantages of the accumulated physical apparatus and increasing its efficiency and accuracy through the use of modern machine learning tools.

Evaluating the Potential of Applying Machine Learning Tools to Metabolic Pathway Optimization

Successful engineering of a microbial host for efficient production of a target product from a given substrate can be viewed as an extensive optimization task. Such a task involves the selection of high activity enzymes as well as their gene expression regulatory control elements (i.e., promoters and ribosome binding sites). Finally, there is also the need to tune expression of multiple genes along a heterologous pathway to relieve constraints from rate-limiting step and help reduce metabolic burden on cells from unnecessary over-expression of high activity enzymes. While the aforementioned tasks could be performed through combinatorial experiments, such an approach incurs significant cost, time and effort, which is a handicap that can be relieved by application of modern machine learning tools. Such tools could attempt to predict high activity enzymes from sequence, but they are currently most usefully applied in classifying strong promoters from weaker ones as well as combinatorial tuning of expression of multiple genes. This perspective reviews the application of machine learning tools to aid metabolic pathway optimization through identifying challenges in metabolic engineering that could be overcome with the help of machine learning tools.

Statistical uncertainty quantification to augment clinical decision support: a first implementation in sleep medicine

Machine learning has the potential to change the practice of medicine, particularly in areas that require pattern recognition (e.g. radiology). Although automated classification is unlikely to be perfect, few modern machine learning tools have the ability to assess their own classification confidence to recognize uncertainty that might need human review. Using automated single-channel sleep staging as a first implementation, we demonstrated that uncertainty information (as quantified using Shannon entropy) can be utilized in a “human in the loop” methodology to promote targeted review of uncertain sleep stage classifications on an epoch-by-epoch basis. Across 20 sleep studies, this feedback methodology proved capable of improving scoring agreement with the gold standard over automated scoring alone (average improvement in Cohen’s Kappa of 0.28), in a fraction of the scoring time compared to full manual review (60% reduction). In summary, our uncertainty-based clinician-in-the-loop framework promotes the improvement of medical classification accuracy/confidence in a cost-effective and economically resourceful manner.

Regularised orthogonal machine learning for nonlinear semiparametric models

Summary This paper proposes a Lasso-type estimator for a high-dimensional sparse parameter identified by a single index conditional moment restriction (CMR). In addition to this parameter, the moment function can also depend on a nuisance function, such as the propensity score or the conditional choice probability, which we estimate by modern machine learning tools. We first adjust the moment function so that the gradient of the future loss function is insensitive (formally, Neyman orthogonal) with respect to the first-stage regularisation bias, preserving the single index property. We then take the loss function to be an indefinite integral of the adjusted moment function with respect to the single index. The proposed Lasso estimator converges at the oracle rate, where the oracle knows the nuisance function and solves only the parametric problem. We demonstrate our method by estimating the short-term heterogeneous impact of Connecticut’s Jobs First welfare reform experiment on women’s welfare participation decision.

Analysis of Machine Learning and Deep Learning Frameworks for Opinion Mining on Drug Reviews

Abstract Opinion mining from medical forums such as health check-ups is sparking growing interest and a stimulating area for natural language processing. This allows for a better understanding of patient health status and drug reactions while generating new knowledge for health care professionals and drug manufacturers, which helps improve the quality of service and produce more effective treatments. In this paper, the researchers present a framework of opinions classification of drug reviews. The objective of this work is to find the best model for analyzing patients’ emotions about drugs. In this sense, the researchers oppose classical text vectorization methods (bag of words, term frequency-inverse document frequency) and word embedding methods (Word2vec, GloVe) for classical opinion mining face to modern machine learning tools with the Convolutional Neural Network (CNN), the Recurrent Neural Networks (Long Short-term Memory and Bidirectional Long Short-Term Memory). Experiments results show that the best model for drug reviews was achieved by CNN based on the Skip-gram model (85% accuracy). Experiments have led to conclude that the performance of a given model will depend on the type of dataset used, on feature representation and better collaboration between classifiers and feature extraction methods.

Features of the implementation of artificial intelligence and digital technologies in industrial production: promising directions and modern trends in automation

The article discusses the problems of optimizing the work of modern industrial production, promising areas of automation, the industrial Internet of things. The author’s principle of optimizing the operation of automated systems through a number of promising technologies is described: a model of an improved abstract executor and distributed computing, modern machine learning tools. The application of information security methods in ensuring the search for the optimal connection between objects and subjects of an automated control system is given. The economic and technological significance and role of the potential described and considered implementations are formalized.

Seeing Beyond the Trees: Using Machine Learning to Estimate the Impact of Minimum Wages on Labor Market Outcomes

We assess the effect of the minimum wage on labor market outcomes such as employment, unemployment, and labor force participation for most workers affected by the policy. We apply modern machine learning tools to construct demographically-based treatment groups capturing around 75% of all minimum wage workers—a major improvement over the literature which has focused on fairly narrow subgroups where the policy has a large bite (e.g., teens). By exploiting 172 prominent minimum wages between 1979 and 2019 we find that there is a very clear increase in average wages of workers in these groups following a minimum wage increase, while there is little evidence of employment loss. Furthermore, we find no indication that minimum wage has a negative effect on the unemployment rate, on the labor force participation, or on the labor market transitions. Furthermore, we detect no employment or participation responses even for sub-groups that are likely to have a high extensive margin labor supply elasticity—such as teens, older workers, or single mothers. Overall, these findings provide little evidence for changing search effort in response to a minimum wage increase.